A proportional-derivative control strategy for varying the restart parameter in GMRES(m)

The Generalized Minimal Residual (GMRES) method is one of the most popular algorithms for the solution of large, sparse and unsymmetric linear systems of equations Ax = b [6]. The idea behind the GMRES is to construct an orthogonal matrix Vk such that its k columns span a Krylov subspace. The time and memory costs of computing Vk are proportional to k, so that, as k grows, the computational cost of GMRES becomes unreasonably high. Thus, in practice, the restarted GMRES method (denoted as GMRES(m)) is used, with m small. In each cycle j, GMRES(m) uses an initial residual as a starting guess for GMRES which constructs a Krylov subspace of dimension m and computes a new residual. Until a stopping criterion is met, GMRES(m) uses the computed new residual as the starting residual for the next cycle, i.e. the next call to a GMRES routine. Normally, at each cycle, the restart parameter m is set to a constant value (practitioners usually set m = 30). However, if the appropriate m is not chosen, the convergence of the GMRES(m) algorithm is not guaranteed [2], and the method may experience stagnation or slow convergence. Unfortunately, it is difficult to know how to choose m a priori and several papers [1, 3, 7, 4, 5, 9] have proposed different adaptive choices of the parameter m. Roughly speaking, these adaptive strategies or rules can be categorized into three groups: (a) simple rules, which however contain empirically chosen parameters that are hard to guess/estimate [7, 4]; (b) rules involving non-trivial calculations of eigenvalues or zeros of polynomials [5, 9] and (c) simple empirical rules [1, 3]. In this paper, we regard the GMRES(m) method as a control problem, in which the parameter m is the controlled variable and propose a new control-inspired strategy for choosing the parameter m adaptively at cycle j, using a discrete proportional-derivative controller which has the form: mj+1 = mj − α ‖rj‖ ‖rj−1‖ + β ‖rj‖ − ‖rj−2‖ 2‖rj−1‖ , (1) where α and β are chosen experimentally. The advantage of this method, named PD-GMRES, is that only a few additional vectors need to be stored and the controller has the capacity to increase the dimension of the Krylov subspace if any convergence problem is detected. Note, however, that empirical parameter choices (of α, β) still need to be made. Numerical comparisons between the proposed PD-GMRES method and other proposals for choosing m ([1, 3]) are presented. The test problems, chosen from the University of Florida’s Matrix Repository [8], are: (1) add20, (2) cdde1, (3) circuit, (4) fpga trans 01, (5) orsirr 1, (6) orsreg 1, (7) pde2961, (8) raefsky1, (9) raefsky2, (10) rdb2048, (11) sherman4, (12) steam2, (13) wang2, (14) watt 1, (15) young3c. For problems with an unspecified right hand side, b